The present invention relates to surveying from recorded images, and more particularly to passive volumetric surveying from pseudo or synthetic stereo images. Thorough understanding, precise measuring, and concise parameterization of the influences of geometric effects on the content of images results in being able to use a compact set of equations to separate the complexities of image geometry from the problem of image correlation—thus paving the way for considerably more efficient automatic image correlation as is the case with traditional coplanar stereo imagery. That, in turn, allow the efficient generation of detailed three dimensional virtual models of the real world as can be seen from the multiple arbitrary viewpoints provided by oblique imagery.
Surveying involves determining the geographic location of points or objects that appear in an image, such as a video image. The images used in surveying may be obtained in any one of a variety of ways. Examples include images taken from a helicopter and images taken from a moving vehicle. A method for obtaining video image data from a moving vehicle is disclosed in commonly assigned U.S. Pat. No. 5,633,946, entitled “METHOD AND APPARATUS FOR COLLECTING AND PROCESSING VISUAL AND SPATIAL POSITION INFORMATION FROM A MOVING PLATFORM” (the '946 patent), which is hereby incorporated by reference. The '946 patent discloses a vehicle with multiple video cameras mounted thereon at different orientations. The vehicle is driven along a street and video images from each of the video cameras are recorded. Each image frame has a time code associated therewith. Accurate spatial position data is also obtained and recorded along with associated time codes. The spatial position data is obtained from a global positioning system (GPS) and an inertial navigation system (INS). The GPS system includes a GPS receiver in the vehicle and a GPS base receiver located in a known position. Errors introduced into the GPS satellite information are eliminated through differential processing of the GPS data gathered by the GPS receiver in the vehicle and the GPS base receiver. The INS comprises a set of rotation and acceleration sensors, and continually measures changes in rotation and changes in velocity as the vehicle moves in latitude, longitude, elevation, pitch, roll and yaw. The INS, therefore, provides raw differential spatial position data with six degrees of freedom between points where precise positional readings are taken by the GPS system.
In post-processing of the above navigation data, Kalman filtering techniques are used to compute a six-degree-of-freedom trajectory for the van as a function of time. This resulting trajectory is a best-fit through all of the navigation data. The indexing of the vehicle's six-dimensional spatial position data by time code allows it to be correlated with each recorded video image, which is also indexed by time code.
For each camera, camera calibration data is also generated and stored for later use in the surveying process. The camera calibration data is intended to adequately represent both the internal optical geometry of the camera and the external six-dimensional location of the camera in the vehicle. The internal aspects relate image pixels with associated rays in camera coordinates. The external aspects relate rays defined in camera coordinates with rays in global coordinates. Both these aspects operate bi-directionally. After both vehicle navigation data post-processing and camera calibration have been completed, a six-dimensional global location can be assigned to each video frame. The combination of the spatial data and the camera calibration data is referred to herein as the image parameters for each frame. After the image parameters have been determined, an image is ready to be used for surveying. Two or more such images at different locations, which view the same object or objects of interest, are used to accomplish the surveying.
Traditionally, aerial surveying has been done using a stereo photogrammetric approach. The mathematics of traditional stereo photography are based on the assumption the source images were taken in approximately the same geometric plane with correction terms for minor deviations of image location from the coplanar assumption. Such an approach must compensate for a relatively short baseline between two cameras by using high resolution digital cameras, which require extraordinarily large amounts of data to be stored. Further, when computing the position of a point or an object which is a great distance from the cameras, as must often be done in the stereo photogrammetric approach, the potential for error in calculating the position of the object is greatly increased.
One objective of the '946 patent is to determine the location of objects seen in two or more images using a generalized approach to surveying in which the locations of objects within images are determined from any two or more images having an overlapping scene, regardless of interimage geometry. It would be desirable to systematically apply this concept to the overlapping pixels of entire images without regard to interimage geometry. However, a difficulty that arises with a plurality of non-coplanar images is that the images are not easily correlated. Correlation algorithms must search through large portions of image data to find pixels in one image that correspond to pixels in the second image. It is commonly known that correlation processes to operate more efficiently on stereo pairs images. For example, in a human-viewable stereo pair, the same object-point in the left and right images lies on the same horizontal line in both images; the difference in lateral displacement along the horizontal line varies with distance of the object-point from the co-plane of the images.